GENERALIZED STATIONARY RANDOM FIELDS WITH 
LINEAR REGRESSIONS - AN OPERATOR APPROACH 



WOJCIECH MATYSIAK AND PAWEL J. SZABLOWSKI 

Abstract. Existence, L 2 -stationarity and linearity of conditional expecta- 
tions E [ATfc | . . . , Xk-2, A'fc-i] of square integrable random sequences X = 
(^fe) fcez satisfying 

oo 

E [X k \. . . , X k _2, X k _i,X k+ i,X k+2 , ■ • ■] = bj + X k+ j) 

3=1 

for a real sequence (bn) ngN , is examined. The analysis is reliant upon the use 
of Laurent and Toeplitz operator techniques. 



1. Introduction 

Bryc J5] examined square integrable stationary random sequences X = (X k ) k&z 
such that for all integer k 

(1.1) E [X k \^ k ] = a (AVi + X k+1 ) , 
where T^ k :— a (Xj : j ^ k), a S M, and 

E[X 2 k \F^ k ] = Q(X k _ 1 ,X k+1 ) 

for some (symmetric) quadratic form Q. The sequences were further analyzed 
in 0, jl()j . |1 1 j and almost complete characterization of their finite dimensional 
distributions is known. Bryc's random fields are related to Hammersley's harnesses 
[Hj, recently revived by Mansuy and Yor [§], and Bryc and Wesolowski (see also 
0] and references therein). 

The first step of the analysis in [3] that led to knowing the distributional structure 
of X was to observe that Ijl.l I) implies that one-sided regressions are linear: 

(1.2) E[X k \F k -. 1 ]=aX k -. 1 , 

where J- k -\ := u (Xj : j < k — 1) and a G K (a is easily identified as the correlation 
coefficient corr(Yfc, X k -i)). 

This paper is devoted to the introductory analysis of random sequences that 
satisfy a generalization of condition Namely, throughout the paper X will 

denote square integrable and normalized random sequence satisfying for all k G Z 

oo 

(1.3) E [X k \^ k ] = bj (X k _j + X k+j ) 
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for some real sequence (&n) ne N (here and further it is assumed that all the series 
involving elements of X converge in L 2 ). We shall formulate conditions for se- 
quences (b n ) under which processes defined by l|1.3|) exist, are L 2 -stationary and 
have linear one-sided regressions: 

oo 

(1.4) K[X k \T< k - 1 ]=Y d ^Xk-j 

i=i 

for some coefficients (/?«)„ related to (b n ) n . (It will turn out that if there exists 
N e N such that bj = for j > N, then = for j > N.) 

Similar problems were examined in a much greater generality by Williams |14j . 
followed by Kingman [J], [H]. In the case of generalized Bryc's random fields, the 
matrix of linear regression coefficients, denoted later as L(b), is a (symmetric) 
Laurent matrix, i.e. doubly infinite matrix, which is constant along the diagonals. 
The structure of the matrix L(b) allows stronger results for the existence of X to 
be obtained than the ones obtained in and [S]. In our analyses, we will also 
refer to Toeplitz matrices, which are infinite (but not doubly infinite) matrices with 
constant diagonals. 

Section [3 is the central section of the paper. Main results of the paper are listed 
in subsections 12 . 21 and 12 .41 while subsections 12 . II and 12 .31 gather the important facts 
from the theory of Laurent and Toeplitz operators needed in further considerations. 
Section |3 contains proofs and auxiliary results. Concluding remarks can be found 
in Section 0] 



2. Main results 



2.1. Laurent matrices and their symbols. Given a sequence of complex num- 
bers {a n )nez, one can construct a Laurent matrix 



a Q a_i a_ 2 



eti ao a_i 



a 2 ai 



ao 



(Throughout the paper, the box indicates the entry in (0, 0) position in the case of 
doubly infinite matrices, or position in the case of doubly infinite vectors.) By 
the classical theorem of Toeplitz |13j . the matrix defines a bounded operator on 
Z 2 (Z) if and only if the numbers (a n ) n are the Fourier coefficients of some function 
a 6 L°° (T) (where T denotes the complex unit circle) 



1 

2^ 



a (e te ) e~ me 



d9, neZ. 



If such a function exists then it is unique and called the symbol of the Laurent 
matrix [a m -„] m n (for a readable introduction to the theory of Laurent and Toeplitz 
operators on l p spaces, see |J). We shall denote by L(a) both Laurent matrix 
[a m _„] m and the bounded operator generated by it. 
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It is known that if a G L°°(T), then the spectrum of the operator L(a) is equal 
to the spectrum IZ(a) of a as an element of Banach algebra L°°(T), which in turn 
is equal to the essential range of a (| • | stands for the Lebesgue measure): 

(2.1) 71(a) = {A G C : \{t G T : \a(t) - X\ < e}\ > Ve > 0}, 

and if ^ 7?,(a), then the inverse of L(a) is the Laurent matrix with symbol a -1 
(Theorem 1.2, 0). 

2.2. Existence and L 2 -stationarity. For a given real sequence (& n ) n eN, consider 
a doubly infinite matrix with (i, j)-th entry (i,j 6 Z) defined as 



(2.2) 



1, if i = j. 



We will denote the symbol of the matrix defined in lj2.2Jl (if it exists) by b, and 
the matrix itself as well as the corresponding operator by L(b). 

Proposition 2.1. If (6 n ) neN is a sequence of real numbers such that the symbol 
b G L°°(T) is positive and ^ 7t(b), then there exist square integrable random 
sequences X = (Xfe) fegZ satisfying <|1.3[) for each k G Z. 

Since the essential range of continuous function a is the image a(T), we see that 
the sufficient condition for the existence of X in the case of b G C(T) is b > 0. 

Remark 2.2. If b\ = a and 6 n = for n > 2, one obtains the case considered by 
Bryc 0. It is easy to verify that b(9) = 1 — 2a cos 9 is positive for 9 G (— it, it] if 
and only if \a\ < |. Since 6 is continuous, we arrive at the assertion of Theorem 1 
HD| (see also 0). 

Remark 2.3. Williams considered, among other things, the problem of the 
existence of square integrable random sequences (Xk) k , with k belonging to an 
arbitrary countable set and satisfying 

(2.3) E [X k \a{X 3 :j^k}]=Y J "i,-V, 

with cifcj > 0, afe j < 1 and the sets {j : a^j > 0} being finite for all k. In the 
case of symmetry of the matrix [akj]k,j, Williams proved that the sequences exist if 
the matrix [dk,j]k,j is invertible and positive definite. Thus Proposition ^ . II extends 
Williams's result to the case of an infinite number of non-zero coefficients in (|2.3|l . 
with the price paid of assuming the special structure of the matrix [cttk,j]k,j- 

Remark 2.4. Kingman [7] considered the problem of the existence of finite sets of 
L 1 random variables satisfying l|2.3|l : in || the problem was extended to random 
sequences under the assumption that the matrix [|afcj|]fcj is transient. 

Proposition 2.5. //X satisfies (|1.3|) for a sequence (b n ) such that b G L°°(T) is 
positive and ^ 7Z(b), then X is L 2 -stationary. 



From now on, we will denote the correlation coefficients of X by r\u = ^LX^Xk 
for k G Z. 
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2.3. Toeplitz matrices with symbols in Wiener algebra. Throughout this 
subsection, we will consider matrices L(b) with symbols in Wiener algebra (see pQ), 
i.e. in the set W = ^(T) of all complex-valued functions a such that 

a(t) = afei fe , where |etfc| < oo and t = exp(i6>) G T. 

kei feez 

W is a Banach algebra with pointwise algebraic operations and the norm ||a|| = 
Sfeezl a *:l- Clearly, W(T) C C(T), so if b G W(T) is positive, the correspond- 
ing random sequence X exists. An important result concerning Wiener algebra is 
Wiener's theorem: if a G T^(T) and a has no zeros on T then a -1 = 1/a G VK(T). 
Denoting by Q (A) the set of all invertible elements of a Banach algebra A, we can 
rephrase Wiener's theorem as 

(2.4) G(W(T)) = {a G W : a(t) ^ V* G T} . 

The reason for considering absolutely summable sequences {b n ) n is that in the 
proof of Theorem 12. 61 one needs to use the Toeplitz operator T(a) generated by an 
absolutely summable sequence (afc)fcgz- 

The Toeplitz matrix (see pQ) defined by a sequence (cfe) fcgZ of complex numbers 
is the infinite matrix 



I Co C_l C_2 

Ci C C_i . . 

C 2 Ci c 

v; ••. ••. ■./ 



It is known that if J^fcez l c fel < 00 1 then [c m _ n ] mj „ induces a bounded operator on 
Z : (N) (Proposition 7.1, 1 ). Analogously to Laurent operators, function 

c(t) = c k t k , t = exp{i9) e T 

is called the symbol of the Toeplitz operator. Also, analogously to the Laurent 
case, we shall denote both the matrix and the operator corresponding to c as T(c). 
By theorems of Gohberg and Duduchava (Theorems 7.3 and 7.4, p*J, if c G W(T), 
then T(c) is invertible on Z 1 (N) if and only if ^ c(T) and wind(c, 0) = 0, where 
wind(c, 0) is the winding number of c with respect to the origin. 

2.4. Linearity of one-sided regressions. 

Theorem 2.6. //X satisfies (|1.3() and < b G W(T), then one-sided regressions 
are linear: 

oo 

(2.5) E[x k \r< k - 1 ]=J2p j x h - j . 

3 = 1 

Coefficients (/3 n ) neN satisfy the following equation: if 

oo 

/?(*) = l-£/5^, P(t) = f3(l/t), teT, 

3 = 1 

and 

oo oo 

(2.6) v = l-2j2bjrj, w = 1-^/3^, 

3 = 1 3 = 1 
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then 
(2.7) 



b{t) = -0(t)0(t), t g T. 



it) 



Remark 2.7. Let (H°°) denote the closed subalgebra of the Banach algebra 
L°° (T) consisting of all functions with vanishing Fourier coefficients with negative 
(positive) indices. It is well known that in general (unlike to the Laurent case), the 
product of two Toeplitz operators is not a Toeplitz operator. Nevertheless, if only 
ai G H°° (so T(oi) is upper triangular), a 2 6 L°° and a 3 G H°° (so T(a 3 ) is lower 
triangular), then 

(2.8) T(a 1 )T{a 2 )T(a 3 )=T{a 1 a 2 a 3 ) 

(see Proposition 1.13 pQ). In view of the above, l|2.7ll can be interpreted as 





/ 1 - 


-bx -b 2 .. 






(I 


-Pi 


-ft 


..A, 




/ 1 





...\ 




-h 


1 -bi .. 


V 





l 


-A 




-A i 







-b 2 - 


-6i 1 .. 




w 








l 






-A -ft. 


1 






V : 




■J 




V 










{ '■ '•■ 






Remark 2. 


8. Note that 


V 


= Var[ 


x k 




and w 


= Var 


-^fc J-<k-i\- 







In the finite case, that means with only a finite number of non-zero elements of 
{bn) n , it is possible to give a more explicit description of the connection between co- 
efficients (b n ) and (0n) n - (I n such case, the symbol b is a trigonometric polynomial 
and L(b) is a band Laurent operator.) 

Corollary 2.9. //X satisfies H1.3|) and N := sup{n G N : 6„ 7^ 0} < 00 7 then ()2.5|) 
holds and sup{n G N : n ^ 0} = iV. Furthermore, 



bi = 







N-l 



010 



N 



(2.9) 



1 + 


# + ...+ 


PV 


0N 


-2 — 010N 


-1 — PiPn-i 




i + P! + 


..+01 


Pi 


- P1P2 - ■ 


■ — Pn-iPn 




1+0I + 


..+0% 




Pn 




1 + 


PI + ...+ 


PV 



&JV-1 
b N 



Remark 2.10. If X satisfies (|1.3|l with b\ — a and b n — for n > 2 (see j3j), then 
by Corollary EHI one gets that E [X k \ F<k-\] = 0iX k -i (cf. Multiplying 
this equation by Xk-i and taking the expectations, yields 0i = r\. Finally, a = 
n/(l + r?) by J221 (see Theorem 3.1 0, and paper [TU]L 

Remark 2.11. In |14j . Williams proved that under the assumptions stated in Remark 
12.31 a variant of l|2.3|l holds with the er-algebra cr{Xj : j ^ k} being replaced by more 
general er-algebras a{Xj : j ^ S}, S C J. Thus, Theorem 12.61 is an extension of 
Williams's result to the case of L 2 -stationary random sequences indexed by integers 
and with an infinite number of non-zero regression coefficients. 
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3. Proofs of main results 

3.1. Existence and L 2 -stationarity. The main observation behind the proof of 
Proposition ^ .H is the following relation of the matrix of linear regression coefficients 
L(b) and the covariance matrix of the random sequence X that can be traced (in 
the finite case) to at least the paper of Kingman [xj; in implicit form, one can find 
it also in ^1] . Namely, if X satisfies H1.3(l and R denotes its (bi-infinite) covariance 
matrix, then 

(3.1) L(b) R = A, 

where A is the (bi-infinite) diagonal matrix with (J, j)-th entry equal to 

(3.2) Vj = E [Xj - E [Xj ] 2 , j G Z. 

It is easy to check that (13.1(1 holds true by multiplying p. 3(1 by Xj, j G Z, and 
calculating the unconditional expectation (see Proposition 2). 

Proof of Provosition \2. 1\ Positivity of b implies positive definitcness of L(b). Con- 
dition ^ guarantees existence of the inverse L _1 (6) = L(6 _1 ), which is 
symmetric and positive definite. Therefore, there exists the centered Gaussian se- 
quence X = {Xk) keIi with the covariance matrix L _1 (6). L _1 (6) is a Laurent 
matrix, hence X is L 2 -stationary. Therefore, all the numbers Vj defined in ((3.2(1 
are equal to a real number v. Since Gaussian sequences have linear regressions, X 
satisfies 1(1.3(1 . □ 

Proof of Provosition \2.5l Matrix L _1 (6) in the proof of Proposition 12 . II is Laurent. 

□ 

In view of Corollary 12.51 we can write the covariance matrix as R = L(r) with 
r G L°°(T), and then 1(3.1(1 reads as L(b)L(r) = vl (here and further, I denotes the 
identity matrix of appropriate dimensions). Since the symbol of the product of two 
Laurent matrices is the product of the symbols, we get that 

(3.3) &(*)K*) =v, t = exp(£0) G T. 

3.2. Linearity of one-sided regressions. Lemma f3. II is needed in the proof of 
Theorem 12. 61 (the theorem imitates the ideas of Williams ^2], for a simple case see 
Q21, Chapter 15.10). 

Lemma 3.1. Let (X, ||-||) be a Banach space and B(X) be the Banach algebra of 
bounded linear operators on X (the operator norm will also be denoted by \\-\\). 

Suppose (y„)„ eN G X and (T n ) neN G B(X). Assume that y n — * y^ in X, 
T n — > Too in B(X) and is invertible. If (x n ) n ^m satisfies T n x n = y n , then 
x n — > £oo in X, where x^ = T" 1 ^. 

Proof. Since for any (non-zero) Banach algebra A with identity, Q{A) is open in A 
(see e.g. [T2], 18.4, Corollary 1), then there exists N G N such that T~ x G B(X) 
for all n > N. For those n, 

\\x n - Zoo || = WT^Vn - T^yooW = \\T~ l y n - T^y n + T^y n - T^-yooW < 

^WT-'-T^WWyJ + WT^Wl^-y^l 

For any Banach algebra A, the mapping x \— > x~ l is a homeomorphism of Q{A) onto 
itself ([I2j,18.4, Corollary 1), so HT" 1 - T^\\ -> 0. Clearly, due to the convergence 
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of {y n )m the sequence (||y n ||)„ is bounded, so the right hand side of the above 
inequality converges to 0. □ 



Proof of Theorem \2.6\ Fix k 6 Z, m £ N and set y n 
n = 1, . . . , m. Since 



[-X"k+ra-l |-7"< fc— l,>fc-f m ) 



oc n— 1 

= ^ ftj-X'fc+Tj-l-j + ^ 6jE [Xk-\- n -l-j \^<k-l 



j=n 



j — 1 j=m— n+1 

for n = 1, . . . , m, we see that the vector y = (y\ . . . y r n) T satisfies the system 
of linear equations T m (b) y = d, with the main matrix 





( 1 -h -b 2 . 


■ -6 m -i\ 




-b x 1 -bi . 


• -b m -2 


T m (b) = 


-b 2 -b x 1 


■ -b m -3 




\— Oro-l —O m -2 —Om-3 


1 / 



and vector d consisting of some infinite linear combinations (convergent in L 2 ) of 
random variables . . . , Xk- 2 , -Xfc-i and X/. +m , Xk+ m +i, ■ ■ •> with some real coeffi- 
cients that do not depend on k. 

We claim that (for sufficiently large m), T m (b) is invertible. Indeed, T m (b) is the 
principal m x m submatrix of the infinite Toeplitz matrix T(b). The symbol b is 
real- valued, continuous and positive (hence wind(fe, 0) = 0), so by the Krein theorem 
(PP, Theorem 1.15), T(b) is invertible on I 2 . Hence, and by the assumption that 
b G C(T), we are in a position to use the Gohberg-Feldman theorem (£Q, Theorem 
2.11). This theorem implies that the sequence of n x n matrices (2~n(6)) ngN is stable, 
which means, among other things, that T n (b) is invertible for sufficiently large n. 

Therefore, solving the system T m (b) y = d, we get, in particular, 



(3-4) yi — E [Xk\J-<k-i,>k+m] = ^ /3j, m Xk-j + ^ jj tm Xk+ m -i+j , 

say, with pj. m and ~fj >m real and not depending on k. Multiplying both of sides 
of (|3.4|) by . . . , Xk-2, Xk-i and Xk+ m , Xk+ m +i, ■ ■ ■, and taking expectations, we 
obtain an infinite system of linear equations with unknowns (3j y m and 7j TO (j G N) , 
which can be written in the matrix form A(m)x(m) = b(m) with 



b(m) = 
x(m) = 

A(m) = 



(• ■ ■ r 2 |ri] r m r m+ i 
(... 02,m 



7l,m 72,m 



Ai A 2 {m] 
A 3 (m) A 4 
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and 



Aa 



A 3 (m) 



1 

r\ 



r\ r 2 
1 n 

W 



A 2 (m) = 



r m +2 r m+ i^ 
r m +3 r m+2 
r m +i r m+3 



r m+3 Tm+2 

Tm+2 1~m+3 

\r m +i r m+2 

1 1 T\ r 2 

T\ 1 T\ 

r 2 ri 1 



Observe that submatrices Ai and A4 do not depend on m. 
By l|3.3|l and by the Wiener theorem, r G VF(T). Therefore 

b(m) — >(■■■ r 2 ...) T =:b 



in V-{1). 



Since II A I 



sup r , 



for A = [a m ,„] 



regarded as an element 



of B (^(Z)) , A m tends in S (Z X (Z)) to A M defined as 

A 



Ai 






A 4 



(here and further, denotes the zero matrix of the appropriate infinite dimensions). 

Now we will show that the inverse of Aoo exists and belongs to B (Z X (Z)). First, 
observe that A4 can be identified in a natural way with an operator acting on Z (N), 
namely with the Toeplitz operator generated by sequence (a/c)fc 6 z whh au = r\u 
for k £ Z, having the symbol r equal to the symbol of the correlation matrix L(r). 
By (|3.3J) . r is real- valued, continuous and positive (hence wind(r, 0) = 0), so by 
the Gohberg and Duduchava theorems, A4 is invertible on I 1 . Analogously, Ai is 
invertible on I 1 . Clearly, A^, 1 £ B (Z 1 (Z)) and is of the form 





I) 



Now we are in a position to use Lemma [3. II with X — Z 1 (Z). Doing so, we get 
that 

x(m) — >A TC 1 b 00 =:x 00 in.Z 1 (Z), 

and 







for some real (3j , j £ N. Note that 



(3.5) 







(I 


r\ 


'2 


^3 




-1 


(n\ 


02 




n 


1 


ri 


r 2 






T2 






T2 


' - i 


1 








T3 






\'- 








'■■j 







On the other hand, by applying Levy's convergence theorem for backward mar- 
tingales to Q3.4|). we deduce that the limit 

00 00 
L := lim ( > fij m Xk-j + > ,7i m Xk+ m +j-i 

m— >oo \* — » * — * 
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exists almost surely and in L l . We claim that L = J2jLi PjXk-j- Indeed, 



E 



Pj,m,Xk-j + 7j,m-^fc+ro+j-l — PjXfr- 
3=1 3=1 3=1 



< 



< E 



^ Pj,rnXk-j - ^2 &i X k-3 

i=i 3=1 



E 



+m+i — 1 



But 

and 

E 



j=i 



3=1 3=1 



3=1 3 = 1 



3 = 1 



3 = 1 



when m tends to infinity, which establishes our claim. 
Thus, by Levy's theorem applied to l|3.4fl . 



E 



P) •7 7 <k-l,>fc+r. 



m— 1 



3 = 1 



Taking the conditional expectation of both sides of the above equation with respect 
to F<k-i, yields 

Since £ H°°, T (j?) T(r) is a Toeplitz matrix by flTBjl . Using 



(3.6) 



T(r) 







ft 


^2 


ft 


= ^3 


W 





which is a direct consequence of <|3.5[) . one can easily verify that the only non- 
zero element of the first row of the matrix T (^f]j T(r) is its first element, which is 

equal to w. In other words, (3 r £ H°°. Since (3 £ H°°, we get that (3 r (3 £ H°°, 
which means, again due to 12.8( 1 . that T 1/3] T{r)T {(3) is a lower triangular Toeplitz 



matrix. But T U3*j T{r)T ((3) is symmetric, hence it must be a diagonal matrix with 

constant diagonal w. Thus, (3{l/i)r{t)j3(t) = w and multiplying it by b and using 
lpO)l . we arrive at (|2~7jl . □ 

Proof of Corollarv \2.iA If supjn £ N : b n ^ 0} = N, then from comparing the 
powers of t in (|2.7() , it follows that for n > N + 1 

oo 
3 = 1 
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which means that the vector (/3jv+i Pn+2 ■ ■ ■) belongs to the kernel of T{0). 
Since T(f3) is invertible, we get that sup{n <E N : /3 n ^ 0} < N. Comparing the 
coefficients at t N at both sides of l|2.7|l . we deduce that /3jv 7^ and v/w = &jv//3jv; 
comparing all the powers of t in (|2.7|l . we get (j2.9(l . □ 



4. Concluding remarks 

Remark 4.1. Random sequences satisfying ljl.3|l are the examples of AR time series; 
equation l|3.6fl is the Yule- Walker equation. 

Remark 4.2. Since one of the assumptions of Theorem 12.61 is the absolute summa- 
bility of the sequence (b n ) n , which in view of Wiener's theorem means the absolute 
summability of the sequence of correlation coefficients of X, one may rephrase the 
assertion of Theorem 12.61 by saying that the sequences with short memory, satis- 
fying (|1.3f) . have linear one-sided regressions. The question of extending Theorem 
12.61 to the case of processes with long memory is left open. 

Remark 4.3 (Conditions for positivity of the symbol b). If N = sup{n 6 N : 
b n 0} < 00, then a lemma of Fejer and Riesz asserts that cosine polynomial 
b(9) = 1 — 2 53n=i b n cos(n6) is non-negative if and only if it can be expressed in 
the form 



b(0) 



N 1 

c„ exp(in6 

n=0 



where 



N N-r 

1 = c 2 j and b r = c j c j+r for r = 1,...,N. 

When the number of non-zero elements of the sequence (b n ) n is infinite, from 
the Szego-Kolmogorov-Krein theorem, under the additional assumption 



/ \ogb(6)d6 > -00, 

J — TT 



b is non-negative if and only if 

&(0) = |C(exp(i0))| 



where 



and 



C(z) = 

n=0 




1 r 

a n = — / cxp (in9) log b(9)du for n — 0, 1, 2, ... . 
27r J-* 

In this case, the relation between coefficients (b n ) n and (c n ) n is rather complicated. 

Nevertheless, it is easy to notice that if only X) n eN l < ^/^' then b is positive. 
(This condition, in the setting of [3], reduces to \a\ < 1/2, see Remark 12. 21 ) 
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